Real-time driver observation and scoring for driver&#39;s education

ABSTRACT

A method includes, during a driving session in which a student driver operates a vehicle, gathering driving skill data indicative of at least one of behavior of the student driver, acceleration of the vehicle, braking of the vehicle, or steering of the vehicle, and gathering comments from a driving instructor about the driving session. The method also includes generating a driving session report corresponding to the driving session. The driving session report includes at least one score based on the driving skill data, and the comments from the driving instructor about the driving session. Still further, the method includes causing the driving session report to be displayed on a display device.

RELATED APPLICATIONS

This application is a continuation application of U.S. application Ser.No. 14/511,750 filed Oct. 10, 2014, which is a continuation applicationof U.S. application Ser. No. 13/844,090, filed Mar. 15, 2013, thedisclosure of which is hereby incorporated herein by reference in itsentirety for all purposes.

TECHNICAL FIELD

The present disclosure generally relates to a system and a method forobserving the driving skills of students in a driver's education classin real-time and, more particularly, to a computer system to collect andanalyze the real-time data to provide a quantitative score of the skillsof each student driver.

BACKGROUND

One of the most important goals of driving instruction is teachingstudent drivers how to drive confidently and safely. Driving instructorsmust teach students how to accelerate, brake, and steer. Drivinginstructors must also teach students how to watch the road ahead whilealso checking the rear and side mirrors. However, during live drivingsessions, it is difficult for a driving instructor to quantify how wella student driver is demonstrating these skills.

SUMMARY

Accordingly, it may be advantageous use a computer device to observe astudent driving during a driving session to gather data about thestudent driver's performance and generate reports based on the gathereddata.

In one embodiment, a method comprises, during a driving session in whicha student driver operates a vehicle, gathering, from one or more sensorsin a mobile device, driving skill data indicative of at least one ofbehavior of the student driver, acceleration of the vehicle, braking ofthe vehicle, or steering of the vehicle, and gathering, from a userinterface, comments from a driving instructor about the driving session.The method further includes generating, by a module speciallyconfiguring a computing device, a driving session report correspondingto the driving session. The driving session report includes at least onescore based on the driving skill data, and the comments from the drivinginstructor about the driving session. Still further, the method includescausing, by the module specially configuring the computing device, thedriving session report to be displayed on a display device.

In another embodiment, a system comprises a user interface, one or moresensors configured to generate driving skill data indicative of at leastone of behavior of a student driver, acceleration of a vehicle, brakingof the vehicle, or steering of the vehicle, a non-transitory memorystoring computer-readable instructions, and one or more processors. Theone or more processors are specially configured by the computer-readableinstructions such that, when executed by the one or more processors, thecomputer-readable instructions cause the one or more processors to,during a driving session in which a student driver operates a vehicle,gather, from one or more sensors, the driving skill data indicative ofat least one of behavior of the student driver, acceleration of thevehicle, braking of the vehicle, or steering of the vehicle, and gather,via the user interface, comments from a driving instructor about thedriving session. The computer-readable instructions further cause theone or more processors to generate a driving session reportcorresponding to the driving session. The driving session reportincludes at least one score based on the driving skill data, and thecomments from the driving instructor about the driving session. Stillfurther, the computer-readable instructions cause the one or moreprocessors to cause the driving session report to be displayed on theuser interface.

In yet another embodiment, a non-transitory computer-readable mediumstores instructions that specially configure one or more processors of acomputer system. When executed by the one or more processors, theinstructions cause the computer system to, during a driving session inwhich a student driver operates a vehicle, gather, from one or moresensors, the driving skill data indicative of at least one of behaviorof the student driver, acceleration of the vehicle, braking of thevehicle, or steering of the vehicle, and gather, via the user interface,comments from a driving instructor about the driving session. Theinstructions also cause the computer system to generate a drivingsession report corresponding to the driving session. The driving sessionreport includes at least one score based on the driving skill data, andthe comments from the driving instructor about the driving session.Still further, the instructions cause the computer system to cause thedriving session report to be displayed on the user interface.

BRIEF DESCRIPTION OF THE DRAWINGS

The figures described below depict various aspects of the system andmethods disclosed herein. It should be understood that each figuredepicts an embodiment of a particular aspect of the disclosed system andmethods, and that each of the figures is intended to accord with apossible embodiment thereof. Further, wherever possible, the followingdescription refers to the reference numerals included in the followingfigures, in which features depicted in multiple figures are designatedwith consistent reference numerals.

FIG. 1 illustrates a block diagram of a computer network, a computerserver, a mobile device, and an on-board computer on which an exemplarydriver's education evaluation system and method may operate inaccordance with the described embodiments;

FIG. 2 illustrates a block diagram of an exemplary mobile device;

FIG. 3 depicts an exemplary driver's education evaluation monitoringmethod for implementing the driver's education evaluation system inaccordance with the presently described embodiments;

FIG. 4 depicts an exemplary primary indicator logging method forimplementing the driver's education evaluation system in accordance withthe presently described embodiments;

FIGS. 5A-B depict an exemplary secondary performance indicator loggingmethod for implementing the driver's education evaluation system inaccordance with the presently described embodiments;

FIG. 6 depicts an exemplary embodiment of a driver's educationevaluation score determination method for implementing the driver'seducation evaluation system in accordance with the presently describedembodiments;

FIG. 7 depicts an exemplary embodiment of a driver's educationevaluation scanning score determination method for implementing thedriver's education evaluation system in accordance with the presentlydescribed embodiments;

FIG. 8-10 are exemplary client applications pages to be displayed on ascreen of a mobile device as part of the user interface used toimplement the driver's education evaluation system in accordance withthe presently described embodiments;

FIG. 11-15 are exemplary reports generated by the driver's educationevaluation system in accordance with the presently describedembodiments.

DETAILED DESCRIPTION

Although the following text sets forth a detailed description ofnumerous different embodiments, it should be understood that the legalscope of the invention is defined by the words of the claims set forthat the end of this patent. The detailed description is to be construedas exemplary only and does not describe every possible embodiment, asdescribing every possible embodiment would be impractical, if notimpossible. One could implement numerous alternate embodiments, usingeither current technology or technology developed after the filing dateof this patent, which would still fall within the scope of the claims.

It should also be understood that, unless a term is expressly defined inthis patent using the sentence “As used herein, the term ‘_(——————)’ ishereby defined to mean . . . ” or a similar sentence, there is no intentto limit the meaning of that term, either expressly or by implication,beyond its plain or ordinary meaning, and such term should not beinterpreted to be limited in scope based on any statement made in anysection of this patent (other than the language of the claims). To theextent that any term recited in the claims at the end of this patent isreferred to in this patent in a manner consistent with a single meaning,that is done for sake of clarity only so as to not confuse the reader,and it is not intended that such claim term be limited, by implicationor otherwise, to that single meaning. Finally, unless a claim element isdefined by reciting the word “means” and a function without the recitalof any structure, it is not intended that the scope of any claim elementbe interpreted based on the application of 35 U.S.C. §112, sixthparagraph.

FIG. 1 illustrates a block diagram of an exemplary driver's educationevaluation system 100. The high-level architecture includes bothhardware and software applications, as well as various datacommunications channels for communicating data between the varioushardware and software components. The driver's education evaluationsystem 100 may be roughly divided into front-end components 102 andback-end components 104. The front-end components 102 are disposedwithin one or more mobile devices 110. The mobile device 110 may bepermanently or removably installed in a vehicle 108 (e.g., a car, truck,etc.). Additionally or alternatively, the vehicle 108 may include anon-board computer 114. The on-board computer 114 may be permanentlyinstalled in a vehicle 108 and may interface with various sensors in thevehicle 108 (e.g., a braking sensor, a speedometer, a tachometer, etc.)and/or may interface with various external output devices in the vehicle108 such as one or more tactile alert systems 120, one or more speakers122, one or more displays (not shown), etc. The on-board computer 114may supplement the functions performed by the mobile device 110described herein by, for example, sending and/or receiving informationto and from the mobile device 110. Alternatively, the on-board computer114 may perform all of the functions of the mobile device 110 describedherein. In such cases, no mobile device 110 may be present in the system100. One or more student drivers 106 may be operating the vehicle 108.The mobile device 110 and on-board computer 114 may communicate with thenetwork 130 over links 112 and 118, respectively. Additionally, themobile device 110 and on-board computer 114 may communicate with oneanother directly over link 116. The vehicle 108 may also include atactile alert system 120 (e.g., a seat that can vibrate) that maypresent tactile alerts to the vehicle operator 106 on command from themobile device 110 and/or the on-board computer 114. While shown in aslightly reclined sitting position, those of ordinary skill in the artwill appreciate that the student driver 106 could be situated in anynumber of ways (e.g., reclining at a different angle, standing, etc.)and operating the vehicle using controls other than the steering wheeland pedals shown in FIG. 1 (e.g., one or more sticks, yokes, levers,etc.). The plurality of mobile devices 110 may be located, by way ofexample rather than limitation, in separate geographic locations fromeach other, including different areas of the same city, differentcities, or different states, and being in mobile vehicles, may move fromone geographic location to another.

The front-end components 102 communicate with the back-end components104 via the network 130. The network 130 may be a proprietary network, asecure public internet, a virtual private network or some other type ofnetwork, such as dedicated access lines, plain ordinary telephone lines,satellite links, combinations of these, etc. Where the network 130comprises the Internet, data communications may take place over thenetwork 130 via an Internet communication protocol. The back-endcomponents 104 include a server 140. The server 140 may include one ormore computer processors adapted and configured to execute varioussoftware applications and components of the driver's educationevaluation system 100, in addition to other software applications. Theserver 140 further includes a database 146. The database 146 is adaptedto store data related to the operation of the driver's educationevaluation system 100. Such data might include, for example, datacollected by a mobile device 110 and/or on-board computer 114 pertainingto the driver's education evaluation system 100 and uploaded to theserver 140 such as images, sensor inputs, data analyzed according to themethods discussed below, or other kinds of data. The server 140 mayaccess data stored in the database 146 when executing various functionsand tasks associated with the operation of the driver's educationevaluation system 100.

Although the driver's education evaluation system 100 is shown toinclude one server 140, one mobile device 110, and one on-board computer114 it should be understood that different numbers of servers 140,devices 110, and on-board computers 114 may be utilized. For example,the system 100 may include a plurality of servers 140 and hundreds ofdevices 110, all of which may be interconnected via the network 130. Asdiscussed above, the mobile device 110 may perform the various functionsdescribed herein in conjunction with the on-board computer 114 or alone(in such cases, the on-board computer 114 need not be present).Likewise, the on-board computer 114 may perform the various functionsdescribed herein in conjunction with the mobile device 110 or alone (insuch cases, the mobile device 110 need not be present). Furthermore, theprocessing performed by the one or more servers 140 may be distributedamong a plurality of servers 140 in an arrangement known as “cloudcomputing.” According to the disclosed example, this configuration mayprovide several advantages, such as, for example, enabling nearreal-time uploads and downloads of information as well as periodicuploads and downloads of information. This may provide for a thin-clientembodiment of the mobile device 110 and/or on-board computer 114discussed herein as well as a primary backup of some or all of the datagathered by the mobile device 110 and/or on-board computer 114.Alternatively, the driver's education evaluation system 100 may includeonly the front-end components 102. For example, a mobile device 110and/or on-board computer 114 may perform all of the processingassociated with gathering data, generating performance reports for thestudent driver 106, storing the performance reports, and sending thereports to the back-end components 104 as discussed herein. As such, thedriver's education evaluation system 100 may be a “stand-alone” system,neither sending nor receiving information over the network 130.

The server 140 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. The controller 155 may include a program memory 160, aprocessor 162 (may be called a microcontroller or a microprocessor), arandom-access memory (RAM) 164, and an input/output (I/O) circuit 166,all of which may be interconnected via an address/data bus 165. Theprogram memory 160 may be configured to store computer-readableinstructions that when executed by the processor 162 cause the server140 to implement a server application 142 and a web server 143. Theinstructions for the server application 142 may cause the server 140 toimplement the methods described herein. While shown as a single block inFIG. 1, it will be appreciated that the server application 142 mayinclude a number of different programs, modules, routines, andsub-routines that may collectively cause the server 140 to implement theserver application 142. It should be appreciated that although only onemicroprocessor 162 is shown, the controller 155 may include multiplemicroprocessors 162. Similarly, the memory of the controller 155 mayinclude multiple RAMs 164 and multiple program memories 160. Further,while the instructions for the server application 142 and web server 143are shown being stored in the program memory 160, the instructions mayadditionally or alternatively be stored in the database 146 and/or RAM164. Although the I/O circuit 166 is shown as a single block, it shouldbe appreciated that the I/O circuit 166 may include a number ofdifferent types of I/O circuits. The RAM(s) 164 and program memories 160may be implemented as semiconductor memories, magnetically readablememories, and/or optically readable memories, for example. Thecontroller 155 may also be operatively connected to the network 130 viaa link 135 and the I/O circuit 166.

Referring now to FIG. 2, the mobile device 110 may include a display202, a Global Positioning System (GPS) unit 206, a communication unit220, a front image capture device 218, a back image capture device 222,an accelerometer array 224, a user-input device 248, a speaker 246, and,like the server 140, a controller 204. Similarly, the on-board computer114 may comprise a display 202, a Global Positioning System (GPS) unit206, a communication unit 220, a front image capture device 218, a backimage capture device 222, an accelerometer array 224, a user-inputdevice 248, a speaker 246, and, like the mobile device 110, a controller204. The mobile device 110 and on-board computer 114 may be integratedinto a single device or one can perform the functions of both. It willbe appreciated that functions performed by either the mobile device 110or the on-board computer 114 may also be performed by the on-boardcomputer 114 in concert with the mobile device 110. The mobile device110 may be either a general-use mobile personal computer, cellularphone, smart phone, tablet computer, other wearable computer (e.g., awatch, glasses, etc.), or a dedicated driver's education evaluationcomputer. The on-board computer 114 may be a general-use on-boardcomputer capable of performing many functions relating to vehicleoperation or a dedicated driver's education evaluation computer.Further, the on-board computer 114 may be installed by the manufacturerof the vehicle 108 or as an aftermarket modification to the vehicle 108.Further, the mobile device 110 and/or on-board computer 114 may be athin-client device which outsources some or most processing to theserver 140.

Similar to the controller 155, the controller 204 includes a programmemory 208, one or more microcontroller or a microprocessor (MP) 210, arandom-access memory (RAM) 212, and an input/output (I/O) circuit 216,all of which are interconnected via an address/data bus 214. The programmemory 208 includes an operating system 226, a data storage 228, aplurality of software applications 230, and a plurality of softwareroutines 234. The operating system 226, for example, may include one ofa plurality of mobile platforms such as the iOS®, Android™ Palm® webOS,Windows® Mobile/Phone, BlackBerry® OS, or Symbian® OS mobile technologyplatforms, developed by Apple Inc., Google Inc., Palm Inc. (nowHewlett-Packard Company), Microsoft Corporation, Research in Motion(RIM), and Nokia, respectively. The data storage 228 may include datasuch as user profiles and preferences, application data for theplurality of applications 230, routine data for the plurality ofroutines 234, and other data necessary to interact with the server 140through the digital network 130. In some embodiments, the controller 204may also include, or otherwise be communicatively connected to, otherdata storage mechanisms (e.g., one or more hard disk drives, opticalstorage drives, solid state storage devices, etc.) that reside withinthe mobile device 110 and/or on-board computer 114.

The GPS unit 206 may use “Assisted GPS” (A-GPS), satellite GPS, or anyother suitable global positioning protocol (e.g., the GLONASS systemoperated by the Russian government) or system that locates the positionthe mobile device 110 and/or on-board computer 114. For example, A-GPSutilizes terrestrial cell phone towers or Wi-Fi hotspots (e.g., wirelessrouter points) to more accurately and more quickly determine location ofthe mobile device 110 and/or on-board computer 114 while satellite GPSgenerally are more useful in more remote regions that lack cell towersor Wi-Fi hotspots.

The front and back image capture devices 218 and 222 may be built-incameras within the mobile device 110 and/or on-board computer 114 and/ormay be peripheral cameras, such as webcams, cameras installed inside thevehicle 108, cameras installed outside the vehicle 108, etc., that arecommunicatively coupled with the mobile device 110 and/or on-boardcomputer 114. The front image capture device 218 may be oriented towardthe student driver 106 to observe the student driver 106 as describedbelow. The back image capture device 222 may be oriented toward thefront of the vehicle 108 to observe the road, lane markings, and/orother objects in front of the vehicle 108. Some embodiments may haveboth a front image capture device 218 and a back image capture device222, but other embodiments may have only one or the other. Further,either or both of the front image capture device 218 and back imagecapture device 222 may include an infrared illuminator 218 i, 222 i,respectively, to facilitate low light and/or night image capturing. Suchan infrared illuminator 218 i, 222 i may be automatically activated whenlight is insufficient for image capturing.

The accelerometer array 224 may be one or more accelerometers positionedto determine the force and direction of movements of the mobile device110 and/or on-board computer 114. In some embodiments, the accelerometerarray 224 may include an X-axis accelerometer 224 x, a Y-axisaccelerometer 224 y, and a Z-axis accelerometer 224 z to measure theforce and direction of movement in each dimension respectively. It willbe appreciated by those of ordinary skill in the art that a threedimensional vector describing a movement of the mobile device 110 and/oron-board computer 114 through three dimensional space can be establishedby combining the outputs of the X-axis, Y-axis, and Z-axisaccelerometers 224 x, y, z using known methods. The GPS unit 206, thefront image capture device 218, the back image capture device 222, andaccelerometer array 224 may be referred to collectively as the “sensors”of the mobile device 110 and/or on-board computer 114. Of course, itwill be appreciated that additional GPS units 206, front image capturedevices 218, back image capture devices 222, and/or accelerometer arrays224 may be added to the mobile device 110 and/or on-board computer 114.

The communication unit 220 may communicate with the server 140 via anysuitable wireless communication protocol network, such as a wirelesstelephony network (e.g., GSM, CDMA, LTE, etc.), a Wi-Fi network (802.11standards), a WiMAX network, a Bluetooth network, etc. The communicationunit 220 may also be capable of communicating using a near fieldcommunication standard (e.g., ISO/IEC 18092, standards provided by theNFC Forum, etc.). Further, the communication unit 220 may use a wiredconnection to the server 140.

The user-input device 248 may include a “soft” keyboard that isdisplayed on the display 202 of the mobile device 110 and/or on-boardcomputer 114, an external hardware keyboard communicating via a wired ora wireless connection (e.g., a Bluetooth keyboard), an external mouse,or any other suitable user-input device. The user-input device 248 mayalso include a microphone capable of receiving user voice input. Asdiscussed with reference to the controllers 155 and 224, it should beappreciated that although FIG. 2 depicts only one microprocessor 210,the controller 204 may include multiple microprocessors 210. Similarly,the memory of the controller 204 may include multiple RAMs 212 andmultiple program memories 208. Although the FIG. 2 depicts the I/Ocircuit 216 as a single block, the I/O circuit 216 may include a numberof different types of I/O circuits. The controller 204 may implement theRAM(s) 212 and the program memories 208 as semiconductor memories,magnetically readable memories, and/or optically readable memories, forexample.

The one or more processors 210 may be adapted and configured to executeany of one or more of the plurality of software applications 230 and/orany one or more of the plurality of software routines 234 residing inthe program memory 208, in addition to other software applications. Oneof the plurality of applications 230 may be a client application 232that may be implemented as a series of machine-readable instructions forperforming the various tasks associated with implementing the driver'seducation evaluation system 100 as well as receiving information at,displaying information on, and transmitting information from the mobiledevice 110 and/or on-board computer 114. The client application 232 mayfunction to implement a stand-alone system or as a system wherein thefront-end components 102 communicate with back-end components 104 asdescribed herein. The client application 232 may includemachine-readable instruction for implementing a user interface to allowa user to input commands to and receive information from driver'seducation evaluation system 100. One of the plurality of applications230 may be a native web browser 236, such as Apple's Safari®, GoogleAndroid™ mobile web browser, Microsoft Internet Explorer® for Mobile,Opera Mobile™, that may be implemented as a series of machine-readableinstructions for receiving, interpreting, and displaying web pageinformation from the server 140 or other back-end components 104 whilealso receiving inputs from the user. Another application of theplurality of applications may include an embedded web browser 242 thatmay be implemented as a series of machine-readable instructions forreceiving, interpreting, and displaying web page information from theservers 140 or other back-end components 104 within the clientapplication 232. One of the plurality of routines may include an imagecapture routine 238 that coordinates with the image capture devices 218,222 to retrieve image data for use with one or more of the plurality ofapplications, such as the client application 232, or for use with otherroutines. Another routine in the plurality of routines may include anaccelerometer routine 240 that determines the force and direction ofmovements of the mobile device 110 and/or on-board computer 114. Theaccelerometer routine 240 may process data from the accelerometer array224 to determine a vector describing the motion of the mobile device 110and/or on-board computer 114 for use with the client application 232. Insome embodiments where the accelerometer array 224 has X-axis, Y-axis,and Z-axis accelerometers 224 x, y, z, the accelerometer routine 240 maycombine the data from each accelerometer 224 x, y, z to establish avector describing the motion of the mobile device 110 and/or on-boardcomputer 114 through three dimensional space. Furthermore, in someembodiments, the accelerometer routine 240 may use data pertaining toless than three axes, such as when determining when the vehicle 108 isbraking.

A user may launch the client application 232 from the mobile device 110and/or on-board computer 114, to access the server 140 to implement thedriver's education evaluation system 100. Additionally, the customer orthe user may also launch or instantiate any other suitable userinterface application (e.g., the native web browser 236, or any otherone of the plurality of software applications 230) to access the server140 to realize the driver's education evaluation system 100.

The server 140 may further include a number of software applications.The various software applications are responsible for generating thedata content to be included in the web pages sent from the web server143 to the mobile device 110 and/or on-board computer 114. The softwareapplications may be executed on the same computer processor as the webserver application 143, or on different computer processors.

In embodiments where the mobile device 110 and/or on-board computer 114is a thin-client device, the server 140 may perform many of theprocessing functions remotely that would otherwise be performed by themobile device 110 and/or on-board computer 114. In such embodiments, themobile device 110 and/or on-board computer 114 may gather data from itssensors as described herein, but instead of analyzing the data locally,the mobile device 110 and/or on-board computer 114 may send the data tothe server 140 for remote processing. The server 140 may perform theanalysis of the gathered data to evaluate the driving performance of thestudent driver 106 as described below. If the server 140 determines thatthe student driver 106 may be impaired, the server 140 may command themobile device 110 and/or on-board computer 114 to alert the studentdriver as described below. Additionally, the server 140 may generate themetrics and suggestions described below based on the gathered data.

FIG. 3 is a flow diagram depicting an exemplary embodiment of a driver'seducation evaluation method 300 implemented by the driver's educationevaluation system 100. More particularly the method 300 may be performedby the mobile device 110 and/or on-board computer 114 and/or the mobiledevice 110 and/or on-board computer 114 in conjunction with the server140. The method 300 may be initiated by a command (block 302). Thecommand may be a user command received by the mobile device 110 and/oron-board computer 114 via the client application 232. Alternatively oradditionally, the command may be sent by the server 140 or may begenerated automatically by the mobile device 110 and/or on-boardcomputer 114 after the meeting of a condition (e.g., the vehicle 108 hasbeen started; the mobile device 110 is within a specified distance fromthe vehicle, a certain time, etc.). Next, the sensors of the mobiledevice 110 and/or on-board computer 114 may be calibrated (block 304).For example the front image capture device 218 may attempt to detect theface and eye(s) of the student driver 106. Calibration may furtherentail adjusting the front image capture device 218 to account for thestudent driver's 106 skin tone, facial characteristics, etc., ambientlight in the vehicle, the background behind the student driver 106, etc.The back image capture device 222 may also be calibrated, such as, toattempt to detect the road in front of the vehicle, identify lanemarkings, and identify other vehicles on the road. Calibration mayfurther entail adjusting the back image capture device 222 to accountfor the color of the road, road conditions (e.g., a wet road from rainor an icy road from snow), the color of lane markings, the time of dayand ambient light, etc. The accelerometer array 224 may also becalibrated. Such calibration may include accounting for constantvibration (e.g., the vibration caused by the engine of the vehicle 108)or other repetitive forces applied to the mobile device 110 and/oron-board computer 114.

After calibration, the mobile device 110 and/or on-board computer 114may begin to collect data about student driver performance using thesensor(s) on the mobile device 110 and/or on-board computer 114 (block306). FIG. 4 is a flow diagram depicting an exemplary embodiment of aprimary performance indicator logging method 400 implemented by thedriver's education evaluation system 100 while gathering data aboutstudent driver performance at block 306. Performance indicators may be aseries of measurements of conditions or characteristics pertaining tostudent driver performance. Accordingly, the front image capture device218, back image capture device 222, and accelerometer array 224, may beused to measure these conditions and characteristics. Such measurementsmay be logged periodically (e.g., every millisecond, every second, etc.)or may be logged conditionally on the occurrence of an event (e.g., aforce of a magnitude above a certain threshold is detected) and storedin data storage 228 as a performance indicator log. Such performanceindicator logs may also include a timestamp to note the time of themeasurement and/or a location stamp to note the GPS coordinates of themeasurement. The driver's education evaluation system 100 may makeperformance indicator logs for primary performance indicators such as:student driver gaze location, vehicle position relative to lanemarkings, vehicle position relative to other vehicles; and accelerationalong the X, Y, or Z axes. The driver's education evaluation system 100may derive secondary performance indicators from the primary performanceindicators such as: student driver gaze fixation, lane deviation,failure to maintain lane centering, time to collision, time to brake,time to react, longitudinal vehicle control, vehicle braking, vehicleacceleration, and lateral acceleration. Both the primary performanceindicator logs and secondary performance indicator logs may be loggedseparately (e.g., a log for gaze location, a log for X axis force, etc.)or may be logged together. These separate or integrated logs may bestored in data storage 228 or may be transmitted to the server 140 viathe network 130 for remote storage.

In order to include a location stamp for each performance log that maybe recorded, the mobile device 110 and/or on-board computer 114 may takea reading from the GPS unit 206 to determine current location of thevehicle 108 (block 402). As discussed above, a location stamp may berecorded for each performance log. Accordingly, it may be advantageousto take a reading from the GPS unit 206 prior to recording anyperformance log. Additionally, with a series of GPS location logs, thevelocity of the vehicle 108 may be determined. Further, if speed limitdata about the route taken is available, a series of GPS location logsand a calculated velocity may be used to make a determination aboutwhether the student driver 106 is maintaining a speed above a minimumspeed limit and/or below a maximum speed limit. Student driver gazelocation may be determined by monitoring the eye or eyes of studentdriver 106 with the front image capture device 218 (block 404). Studentdriver gaze location may be logged as the horizontal and verticalcoordinates of the student drive's apparent gaze. Student driver gazelocation may be used to determine when the student driver 106 is lookingat the road, mirrors, the dashboard, stereo or air conditioningcontrols, a mobile device, etc. In some embodiments, the clientapplication 232 may log whether the student driver 106 is looking at adistraction (e.g., the stereo) or in the direction of an important areafor vehicle operation (e.g., the road, mirrors, etc.). The driver'seducation evaluation system 100 may differentiate between the importantareas for vehicle operation in gaze location logs. The driver'seducation evaluation system 100 may include a first value in the gazelocation log indicating that the student driver was looking at the road,a second value in the gaze location log indicating that the studentdriver was looking at the rear view mirror, a third value in the gazelocation log indicating that the student driver was looking at the leftside mirror, a fourth value in the gaze location log indicating that thestudent driver was looking at the right side mirror, and a fifth valuein the gaze location log indicating that the vehicle was looking at thedashboard gauges (e.g., speedometer). The client application 232 mayalso include a timestamp and/or location stamp in the gaze location log.If a gaze location log is made every time the student driver startslooking at a different object, then the duration of a particular studentdriver gaze can be determined by the difference between the time thestudent driver 106 began looking at the object and the time the studentdriver 106 begins looking at another object.

The back image capture device 222 may be used to monitor conditions onthe road including identifying lane markings and/or other vehicles onthe road. Vehicle position relative to lane markings may be determinedby processing an image or series of images captured by the back imagecapture device 222 to determine the distance of the vehicle 108 fromlane markings on either or both sides of the vehicle 108 (block 406).The mobile device 110 and/or on-board computer 114 may determine vehicleposition relative to lane markings regularly with a timestamp and/orlocation stamp and store the log of vehicle position relative to lanemarkings in data storage 228 or send the log of vehicle positionrelative to lane markings to the server 140 for remote storage.Similarly, vehicle position relative to other vehicles (also referred toas vehicle headway distance) may be determined by processing an image orseries of images captured by the back image capture device 222 todetermine the distance of the vehicle 108 from other vehicles on theroad in front of the vehicle 108 (block 408). For example, the imagescaptured by the back image capture device 222 may be analyzed to comparethe visual area of an object in front of the vehicle in one or moreimages (i.e., if the visual area is larger in a first image relative toa second image, the object was likely closer at the time the secondimage was captured whereas if the visual area of the object is smallerin a first image relative to a second image, the object was likelyfurther away at the time the second image was captured) and/or thevisual area of the road between the vehicle 108 and an object (i.e., ifthe visual area of the road is larger in a first image relative to asecond image, the object was likely further away at the time the secondimage was captured whereas if the visual area of the road is smaller ina first image relative to a second image, the object was likely closerat the time the second image was captured). Additionally oralternatively, if the back image capture device 222 is properlycalibrated, a single image of the road ahead of the vehicle may besufficient to estimate the distance of the vehicle 108 from the vehicleahead using known trigonometric principles. The mobile device 110 and/oron-board computer 114 may determine vehicle position relative to othervehicles regularly with a timestamp and store the log in data storage228 or send the log to the server 140 for remote storage. Additionally,information from the GPS unit 206 may be incorporated into the log toadd information about the current velocity and/or location of thevehicle 108.

The accelerometer array 224 may be used to monitor forces on the vehiclein the X, Y, and/or Z axis and create accelerometer logs (block 410). Insome embodiments, the Y-axis may be oriented along left to right axis ofthe mobile device 110 and/or on-board computer 114, the Z-axis may beoriented along the top to bottom axis of the mobile device 110 and/oron-board computer 114, and the X-axis may be oriented along the front toback axis of the mobile device 110 and/or on-board computer 114.However, the axes could be oriented in any number of ways. The mobiledevice 110 and/or on-board computer 114 may determine the magnitude of aforce along one of the axes and make an accelerometer log with atimestamp and/or location stamp in data storage 228 or send theaccelerometer log to the server 140 for remote storage.

The user input device 248 may be used to collect comments from thedriving instructor during the driving session (block 412). By activatinga control (for example the “Create Note” button 1004 of FIG. 10), thedriving instructor may create a comment or note about the drivingsession. The driving instructor may record an audible comment or type atextual comment. For example, the driving instructor may make a commentafter instances of poor driving performance such as the student driver106 stopping in a crosswalk, crossing the center line of the street,applying the brakes too late, etc. The driving instructor may also makecomments after instances of good driving performance such as a wellexecuted turn. The comments may be related to a particular action (e.g.,a particularly well executed turn) or general comments about the drivingsession (e.g., “Sam has shown a marked improvement since the sessionlast week.”). The instructor may make notes during and/or after thedriving session. Comments may be entered by the instructor directly onthe first mobile device 110 and/or on-board computer 114 performing thestudent driver performance evaluation methods discussed herein, orcomments may be entered via a second mobile device 110 in communicationwith the first mobile device 110 and/or on-board computer 114. Forexample, the driving instructor may use a tablet computer to recordcomments which are relayed to the first mobile device 110 and/oron-board computer 114 via Bluetooth® or Near Field Communication or anyknown short-range networking technique. Alternatively, the second mobiledevice 110 may communicate with the first mobile device 110 and/oron-board computer 114 via the network 130. The comments may be storedwith a timestamp and/or location stamp in data storage 228 or sent tothe server 140 for remote storage

If the student driver session has concluded, the primary performanceindicator logging method 400 may end. However if the student drivingsession has not concluded, the primary performance indicator loggingmethod 400 may continue to gather data (block 414).

FIGS. 5A-B are flow diagrams depicting an exemplary embodiment of asecondary performance indicator logging method 500 implemented by thedriver's education evaluation system 100 while gathering data aboutpotential student driver performance at block 306. Referring to FIG. 5A,student driver gaze fixation may be determined by analyzing a set ofstudent driver gaze location logs and determining the length of time inwhich the student driver 106 is looking at a particular place (block502). It will be understood that when looking at a particular place, astudent driver 106 may move his or her eyes slightly. Such minorvariations may be disregarded subject to a sensitivity setting. Studentdriver gaze fixation records instances where a student driver has lookedat the same object for more than a threshold period of time (e.g., 100ms) (block 505). For example, student driver gaze fixation may be usedto detect when the student driver 106 has his or her gaze fixed on theroad above a threshold level (e.g., the student driver 106 has notlooked at mirrors or dashboard in more than two seconds). Additionallyor alternatively, student driver gaze fixation may be determined byanalyzing a set of student driver gaze location logs and determining theeye movement of the student driver 106 by calculating the degree towhich the student driver's 106 eyes have moved in a first image relativeto a second image. When employing such an eye movement velocity-basedgaze detection algorithm, student driver gaze fixation may recordinstances where the velocity of eye movement is below a threshold value(e.g., thirty-five degrees per second). If student driver gaze fixationis detected, the client application 232 may make a gaze fixation logwith a timestamp and/or location stamp (block 506).

With the logs of vehicle position relative to lane markings, lanedeviation may be determined by analyzing the logs of vehicle positionrelative to lane markings to determine when the distance between a lanemarking and vehicle 108 indicates that the vehicle 108 has changed lanes(block 508). While lane changes are a normal aspect of vehicleoperation, a slow lane change may indicate that the operator 106 is notproperly controlling the vehicle 108 and/or is distracted. Accordingly,the driver's education evaluation system 100 may analyze the log ofvehicle position relative to lane markings to detect lane changes thatoccur over a period of time greater than a threshold value (e.g., twentyseconds, thirty seconds, etc.) (block 510). When a slow lane deviationis detected, the client application may make a slow lane deviation logwith a timestamp and/or location stamp (block 512).

With the logs of vehicle position relative to lane markings, failure tomaintain lane centering may be determined by analyzing the logs ofvehicle position relative to lane markings to determine when thedistance between a lane marking and vehicle 108 indicates that thevehicle 108 is not centered in the lane (block 514). Similarly to lanedeviation, if a vehicle 108 starts to veer from the center of the laneover a long period of time, this may indicate that the student driver106 is not properly controlling the vehicle 108 and/or is distracted.Accordingly, the driver's education evaluation system 100 may analyzethe log of vehicle position relative to lane markings to detect anincreasing failure to maintain lane centering that occurs over a periodof time greater than a threshold value (e.g., fifteen seconds) (block516). When a failure to maintain lane centering is detected, the clientapplication 232 may make a log with a timestamp and/or location stamp(block 518).

Referring now to FIG. 5B, with the logs of vehicle position relative toother vehicles, time to collision may be determined by analyzing thelogs of vehicle position relative to other vehicles to determine when adecreasing time to collision indicates that the vehicle 108 may be tooclose behind another vehicle (block 520). Time to collision may becalculated in a number of ways (e.g., by dividing the distance betweenthe vehicle 108 and the vehicle ahead by the difference in velocitybetween the two vehicles, etc.). Next, the client application 232 maydetermine the visual area of an object in front of the vehicle 108 in afirst image, determine the visual area of the object in a second image,and calculate the difference between the two areas. Then, the time tocollision may be estimated by noting the change in the differencebetween the two areas relative to the amount of time between the firsttime and the second time. Additionally or alternatively, the clientapplication 232 may determine the visual area of the road in front ofthe vehicle 108 in a first image, determine the visual area of the roadin a second image, and calculate the difference between the two areas.Then, the time to collision may be estimated by noting the change in thedifference between the two areas relative to the amount of time betweenthe first time and the second time. Alternatively, the distance betweenthe vehicle 108 and the vehicle ahead may be calculated with a singleimage using trigonometry as discussed above. Input from the GPS unit 206may be used to determine current velocity of the vehicle 108. Thedriver's education evaluation system 100 may analyze the log of vehicleposition relative to other vehicles to detect when time to collisiondecreases below a threshold value (e.g., 2 second etc.), which mayindicate, for example, that the student driver 106 is tailgating thevehicle ahead (block 520). When a below threshold time to collision isdetected, the client application 232 may make a time to collision belowthreshold log with a timestamp and/or location stamp (block 522).Alternatively or additionally, the data used to calculate time tocollision may also be used to calculate similar metrics such as time tobrake (i.e., the amount of time the student driver 106 has to apply thebrakes in order to prevent collision with an object) and/or time toreact (i.e., the amount of time a student driver 106 has to recognize animminent collision and react to prevent it by swerving and/or applyingthe brakes). In addition to the data used to calculate time tocollision, it may be advantageous to incorporate additional data intothe calculation of time to brake and time to react such as the stoppingcapability of the vehicle 108, road conditions (e.g., wet, icy, unpaved,etc.), and the reaction time of the student driver 106 (e.g., determinedby a test performed on the individual student driver 106, calculated byadjusting average human reaction time to account for the studentdriver's 106 age, health, performance level as determined herein, etc.).As with time to collision, time to brake and/or time to react may becompared to a threshold time and used to generate a performance log.

With the accelerometer logs, vehicle braking or deceleration may bemonitored by noting deceleration sensed by an accelerometer oriented inthe fore-aft direction of the vehicle (i.e., the X-axis) (block 526). Ifthe force measured by the accelerometer array 224 indicates that thebrakes of the vehicle 108 have been applied sharply (e.g., the forcemeasured in the X-axis exceeds a threshold value) (block 528), theclient application 232 may make a hard brake log with a timestamp and/orlocation stamp (block 530).

With the accelerometer logs, vehicle acceleration may be monitored bynoting acceleration sensed by an accelerometer oriented in the fore-aftdirection of the vehicle (i.e., the X-axis) (block 532). If the forcemeasured by the accelerometer array 224 indicates that the acceleratorof the vehicle 108 has been applied sharply (e.g., the force measured inthe X-axis exceeds a threshold value) (block 534), the clientapplication 232 may make a sharp acceleration log with a timestampand/or location stamp (block 536).

With the accelerometer logs, vehicle lateral acceleration may bemonitored by analyzing forces measured by an accelerometer orientedalong the left to right side of the vehicle 108 (i.e., the Y-axis)(block 538). If the force measured by the accelerometer array 224indicates that the vehicle 108 has swerved (e.g., the force measured inthe Y-axis exceeds a threshold value) (block 540), the clientapplication 232 may make a swerve log with a timestamp and/or locationstamp (block 542).

In embodiments where the mobile device 110 and/or on-board computer 114is a thin client device, the mobile device 110 and/or on-board computer114 may send the logs to the server 140 soon after logging the recordedinformation. In such embodiments, the server 140 may analyze the logs ofprimary performance indicators as discussed above to determine secondaryperformance indicators.

Referring again to FIG. 3, after gathering primary and secondaryperformance indicators, the driver's education evaluation system 100 mayanalyze the primary and secondary performance indicators to generate oneor more reports about the driving session (block 308).

Generating the one or more reports may include generating individualskill performance score(s) for the driving session for one or more agaze location score, a scanning frequency score, a gaze location score,a lane deviation score, a lane centering score, a time to collisionscore, a braking score, an acceleration score, or a steering score.Additionally or alternatively, the report may include one or morecomposite score(s) calculated using the individual skill performancescore(s) that were generated. FIGS. 6 and 7 describe examples of howvarious individual skill performance scores and composite scores may begenerated in exemplary embodiments. Further, the report(s) generated mayinclude a recitation of important events that occurred during thedriving session such as braking, acceleration, and/or swerving withforces exceeding a safety threshold (e.g., a hard brake at theintersection of Clark St. and Division St., a hard right turn at theintersections of LaSalle St. and Ohio St., etc.) with a timestamp and/orlocation stamp. The report may also include any comments made by thedriving instructor during and after the driving session. It may beadvantageous to represent these important events and comments visuallyon a map. After the report(s) have been generated, the report(s) may bedistributed and/or displayed (block 310). The report(s) may bedistributed via email or other type of electronic messaging to thedriving instructor, the student driver, and/or the parents of thestudent driver. The report(s) may also be displayed on the display 202of the mobile device 110 and/or on-board computer 114 or any othercomputing device being used by one or more of the student driver,parents, and driving instructor. Additionally, it may be advantageous toprepare a printed version of the report(s) and/or generate automatedvoicemail messages, and using known techniques. The generation anddisplay of the report(s) is discussed in more detail below with respectto the example screenshots FIGS. 11-14. Such analysis, display, anddistribution may be performed using the mobile device 110 and/oron-board computer 114, the server 140, or a combination.

FIG. 6 is a flow diagram depicting an exemplary embodiment of a driver'seducation evaluation score determination method 600 implemented by thedriver's education evaluation system 100 while generating one or morereports about the driving session at block 308. The method 600 firstdetermines an acceleration score using one or more performance logs(block 602). Acceleration score may be determined by subtracting pointsfrom a total score of 100 every time the student driver 106 applies toomuch acceleration during the driving session. The number of pointssubtracted may be determined according to a series of threshold values.For example, 1 point may be subtracted for a “hard” acceleration (e.g.,acceleration above threshold A1 m/s²), 2 points may be subtracted for a“very hard” acceleration (e.g., acceleration above threshold A2 m/s²),and 3 points may be subtracted for a “severe” acceleration (e.g.,acceleration above threshold A3 m/s²). The method 600 may also determinea braking score using one of more performance logs (block 604). Brakingscore may be determined by subtracting points from a total score of 100every time the student driver 106 applies too much braking (ordeceleration) during the driving session. The number of pointssubtracted may be determined according to a series of threshold values.For example, 1 point may be subtracted for a “hard” braking (e.g.,braking above threshold B1 m/s²), 2 points may be subtracted for a “veryhard” braking (e.g., braking above threshold B2 m/s²), and 3 points maybe subtracted for a “severe” braking (e.g., braking above threshold B3m/s²). The method 600 may also determine a steering score using one ormore performance logs (block 606). The number of points subtracted maybe determined according to a series of threshold values. For example, 1point may be subtracted for a “hard” turning (e.g., turning abovethreshold T1 m/s²), 2 points may be subtracted for a “very hard” turning(e.g., turning above threshold T2 m/s²), and 3 points may be subtractedfor a “severe” turning (e.g., turning above threshold T3 m/s²). Ofcourse, it will be understood that for each score different numbers ofpoints may be subtracted and different units may be used. Further, itwill also be understood that the scores may be calculated using methodsother than subtraction from 100 such as adding points to a store forevery acceleration, brake, and/or turn that does exceed the thresholdsdiscussed above.

After determining scores for acceleration, braking, and steering asdiscussed above, the method 600 may multiply each score by a weightingfactor 608 a, b, c. For example, if each score is weighted equally, theweighting factors 610 a-c may all be 0.333. However, it may beadvantageous to weight one score higher than another. For example, sharpacceleration may be less important than braking and steering inevaluating the performance of the student driver 106. In such anembodiment, the weighting factors 610 a-c may be 0.25, 0.35, and 0.40,respectively. In some embodiments, the weighting factors may be adjustedbased on previous data for the user or for a large group of users. Theweighting factors may be adjusted by one of the many known learningalgorithms such as a support vector machine (SVM). The method 600 maythen sum the weighted scores to determine a composite driving sessionscore (block 610). The composite driving session score may be logged inwith a timestamp and stored in data storage 228 and/or sent to theserver 140 for remote storage. Alternatively, it will be understood thatinstead of a weighted sum adding up to a composite driving sessionscore, the client application 232 may instead be a weighted sum that issubtracted from a maximum composite driving session score. An example ofa display with the individual skill scores and composite driving sessionscore may be seen in FIG. 12.

While the exemplary embodiment discussed above uses a 100 point scale,it will be appreciated that a 100 point scale is just one of many pointscales that could be used (e.g., 50 point scale, 200 point scale, 500point scale, 1000 point scale, etc.). Additional primary and secondaryperformance indicators may be used in the determination of the compositedriving session score. For example, a gaze location score, a scanningfrequency score, a gaze location score, a lane deviation score, a lanecentering score, or a time to collision score may be added to thecalculation of the composite driving session score. Each primary andsecondary performance indicator may be used to generate a respectivescore similar to the scores described in connection to FIG. 6. Forexample, a respective score for each may be calculated by subtracting 1point from a total score of 100 for every instance of a gaze fixation,slow lane deviation, failure to maintain lane centering, below thresholdtime to collision, respectively. Once a score for some or all of thegaze fixation, slow lane deviation, failure to maintain lane centering,below threshold time to collision performance has been calculated,scores may be added to the weighted sum discussed above. It will beappreciated that when additional scores are added to the weighted sum,it may be advantageous to change the weighting coefficient for some orall of the other scores in the weighted sum. Additionally, the driver'seducation evaluation system 100 may be configurable to adjustsensitivity, change point deduction values, etc.

FIG. 7 is a flow diagram depicting an exemplary embodiment of a driver'seducation evaluation scanning score determination method 700 implementedby the driver's education evaluation system 100 while generating one ormore reports about the driving session at block 308. The method 700 maydetermine a gaze location distribution using the gaze locations logs(block 702). Because each gaze location log may include geographiccoordinates corresponding to where the student driver 106 was looking ata particular time during the driving session, it may be advantageous togroup the gaze location logs according to their coordinates. Forexample, it is likely that many gaze location logs were made when thestudent driver 106 was looking at one of the mirrors or the road ahead.Thus, even without knowledge of the interior layout of the vehicle 108,the method 700 may be able to determine where important features of thevehicle 108 are located based on the statistical distribution of gazes.A significant number of gazes to the far right may correspond to theright side mirror, a significant number of gazes to the upper center maycorrespond to the rear view mirror, etc. Accordingly, the method 700 maysort the gaze location logs into groups for right side mirror gazes,left side mirror gazes, rear mirror gazes, road gazes, and other gazes.The method 700 may be able to total the amount of time the studentdriver 106 was looking at each location and determine an average amountof time the eyes of the student driver 106 were not focused on the roadahead.

Using the gaze location distribution, the method 700 may calculate amirror checking score (block 704). The mirror checking score may bedetermined by comparing the amount of time during the driving sessionthat the student driver 106 spent gazing at the right side mirror, leftside mirror, and rear view mirror to an expected amount of time. Theexpected amount of time may be a threshold level established by, forexample, observing good drivers to determine how often each of the gooddrivers gaze at each mirror over a period of time (e.g., looking at eachmirror for 0.5 second every 30 seconds). The mirror checking score maybe calculated by subtracting points from 100 every time the studentdriver 106 fails to look at each mirror periodically at the expectedamount during the driving session. Alternatively, using a dataset ofdriving performances by a large number of student drivers (e.g., otherstudent drivers that have used the systems and methods described hereinin the past), the driver's education evaluation system 100 may calculatea distribution of mirror checking performance. Using this distribution,the method 700 may calculate in which percentile the performance of thestudent driver 106 belongs, and store that percentile as the mirrorchecking score.

The method 700 may also determine a gaze fixation score using one ormore performance logs (block 706). The gaze fixation score may bedetermined by subtracting 1 point from a total score of 100 every timegaze fixation is detected during a certain period of time. As with themirror checking score discussed above, using a dataset of drivingperformances by a large number of student drivers (e.g., other studentdrivers that have used the systems and methods described herein in thepast), the driver's education evaluation system 100 may calculate adistribution of gaze fixation performance. Using this distribution, themethod 700 may calculate in which percentile the performance of thestudent driver 106 belongs, and store that percentile as the gazefixation score.

The method 700 may also determine a scanning frequency score using oneor more performance logs (block 708). Scanning frequency score can bedetermined by subtracting 1 point from a total score of 100 every timethe student driver 106 fails to shift his or her gaze from one importantarea for vehicle operation (e.g., the road, mirrors, etc.) to anotherimportant area for vehicle operation within a threshold period of time(e.g., 5 seconds) within a certain period of time. For example, astudent driver 106 who is distracted may not look from the road to checkthe mirrors and speed indicator with sufficient frequency. As with themirror checking score discussed above, using a dataset of drivingperformances by a large number of student drivers (e.g., other studentdrivers that have used the systems and methods described herein in thepast), the driver's education evaluation system 100 may calculate adistribution of scanning frequency performance. Using this distribution,the method 700 may calculate in which percentile the performance of thestudent driver 106 belongs, and store that percentile as the scanningfrequency score. Each score may be logged in with a timestamp and storedin data storage 228 and/or sent to the server 140 for remote storage.

FIGS. 8-10 depict client application pages or screens that may bedisplayed on the display 202 of the mobile device 110 as part of theuser interface used to implement the driver's education evaluationsystem 100. While FIGS. 8-10 depict client application pages or screensbeing displayed on the display 202 of the mobile device 110, it will beunderstood that the client application pages or screens could bedisplayed on the display 202 of the on-board computer 114 in addition tobeing displayed on the mobile device 110 or as an alternative. Theclient applications or pages may be generated by the mobile device 110or sent to the mobile device 110 by the server 140 (e.g., as with a thinclient). The user may launch the client application 232 from the mobiledevice 110 via any suitable manner, such as touch-selecting a clientapplication icon (not shown) on the display 202 of the mobile device 110or speaking a voice command into the microphone (not shown) of themobile device 110. After the user launches the client application 232,the client application 232 may begin to run on the mobile device 110 asdescribed above in connection to block 302.

With reference now to FIG. 8, a home screen 800 of the clientapplication 232 may be displayed on the display 202 of the mobile device110. The home screen 800 may include a “Calibrate” button 802, a “Start”button 804, a “Settings” tab 806, and a “Report” tab 808. When the userselects the calibrate button 802 the client application 232 may executea calibration routine as described above in connection to block 304.

With reference now to FIG. 9, a calibration screen 900 of the clientapplication 232 may be displayed on the display 202 of the mobile device110 during a calibration routine executed in connection to block 304.The calibration screen 900 may include a face detection indicator 902,eye detection indicator 904, the “Cancel” button 906, and a calibrationprogress indicator 908. While the client application 232 is executingthe calibration routine discussed in connection to block 304, thecalibration screen 900 may display a face detection indicator 902showing on the display 202 the visual area perceived by the clientapplication 232 to be the face of the user 106 and/or an eye detectionindicator 904 showing on the display the visual area perceived by theclient application 232 to be an eye of the user 106. If a user selectsthe cancel button 906, calibration may be terminated. A calibrationprogress indicator 908 may display an approximate indication of thestatus of the calibration routine.

Referring again to FIG. 8 when the user selects the “Start” button 804,the client application 232 may begin to collect data during the drivingsession in connection to block 306. With reference now to FIG. 10,driving session data collection screen 1000 may be displayed on thedisplay 202 of the mobile device 110 executed in connection with block306. The driving session data collection screen 1000 may include a“Stop” button 1002. If the “Stop” button 1002 is selected by the user,the vehicle operator impairment monitoring system 100 may terminateoperator impairment monitoring. Selecting the “Stop” button 1002 mayalso permit the user to save additional information about the trip suchas additional instructor comments. The driving session data collectionscreen 1000 may also include a “Create Note” button 1004. If the drivinginstructor activates the “Create Note” button, the instructor may beable to enter a textual and/or audible comment pertaining to the drivingsession as discussed in connection to block 412.

Now referring to FIG. 11, another example screenshot 1100 illustrates areport map including driving session data. The screenshot 1100 may begenerated using the techniques discussed in connection to block 308 anddisplayed using the techniques discussed in connection to block 310. Themap includes a visual representation of the route 1102 taken by thestudent driver 106 during the driving session. At various points on theroute may be projected icons indicating the occurrence of importantevents that occurred during the driving session such as those detectedin connection to block 306 (e.g., hard braking events, sharp turns,etc.). These icons may be used to indicate good driving (e.g., drawingthe route 1102 in green) and instances of less than good driving bydegree. For example, a yellow flag 1104 may indicate “fair” driving(e.g., a “hard” brake as discussed above), an orange flag 1106 mayindicated “poor” driving (e.g., a “very hard” brake as discussed above),and a red flag 1108 (e.g., a “severe” brake as discussed above).Additionally, location-based comments may be indicated with a commentflag 1110. The route 1102 and various icons 1104-1110 may be projectedonto a map in the appropriate place using the location stamps includedwith the performance logs as discussed above.

Now referring to FIG. 12, another example screenshot 1200 illustrates anumerical report for a driving session. The screenshot 1200 may begenerated using the techniques discussed in connection to block 308 anddisplayed using the techniques discussed in connection to block 310. Inparticular, the screenshot 1200 may be generated using the techniquesdiscussed in connection to FIG. 6. The numerical report may be generatedbased on a single driving session or multiple driving sessions as shownin FIG. 12. The numerical report may include information about thestudent driver 1202 (e.g., name, an indicator of how many drivingsessions are included in the numerical report, a photograph, etc.) andinformation about the driving session 1204 (e.g., total duration ofdriving sessions, the number of trips included in the numerical report,and a total distance of driving sessions). The numerical report mayinclude a composite score 1206, an acceleration skill score 1208, abraking skill score 1210, and/or a cornering skill score 1212.Additionally, it may be advantageous to display additional skill scoresand indicators such as a lane maintenance score or time to collisionscore discussed above.

Now referring to FIG. 13, another example screenshot 1300 illustrates avisual mapping of gaze location distribution from a driving session. Thescreenshot 1300 may be generated using the techniques discussed inconnection to block 308 and displayed using the techniques discussed inconnection to block 310. In particular, the screenshot 1300 may begenerated using the techniques discussed in connection to block 702 ofFIG. 7. The visual mapping of gaze location distribution may be drawn asa “heat map” showing the density of the statistical distribution of gazelocations using a progressive color palette. For example, white mayindicate little to no density, yellow may indicate low density, orangemay indicate medium density, and red may indicate high density (i.e.,where the student driver 106 was looking most often). The visual mappingof gaze location distribution may include distributions associated withthe student driver looking at the road 1302, looking at the rear viewmirror 1304, looking at the left side mirror 1306, and/or looking at theright side mirror 1308. The visual mapping of gaze location distributionmay indicate suggestions for how the student driver 106 may includeperformance. For example, in the screenshot 1300, the distribution 1308does not show any orange or red, indicating that the student driver 106may not be checking the right side mirror enough. Accordingly, thedriving instructor may advise the student driver 106 to check the rightside mirror during the next driving session.

Now referring to FIG. 14, another example screenshot 1400 illustrates ascanning score report from a driving session. The screenshot 1400 may begenerated using the techniques discussed in connection to block 308 anddisplayed using the techniques discussed in connection to block 310. Inparticular, the screenshot 1400 may be generated using the techniquesdiscussed in connection to FIG. 7. The scanning score report may includea score 1402 for checking mirrors, a gaze fixation score 1404, ascanning score 1406 (e.g., a scanning frequency score), and/or anindication 1408 of the average amount of time the student driver 106spent with his or her eyes off the road.

Using the systems and methods discussed above, a driving instructor maytake one or more student drivers 106 on one or more driving sessions.Data may be gathered during each driving session and reports may begenerated for each driving session as discussed above. The drivinginstructor may take the same student driver 106 out for multiple drivingsessions over the course of a driving class (e.g., one driving sessionper weekly class). The data and reports associated with each drivingsession may be used to evaluate the progress of the student driver overthe course of the driving class, in particular to determine whether theskills of the student driver are improving. The data and reportsassociated with each driving session may also be used to advise thestudent driver on which skills he or she needs to practice betweendriving classes. Further, the driving instructor may use the systems andmethods discussed above to evaluate a plurality of students in thedriving class. The data and reports associated with each driving sessionmay be used to evaluate the students in the class relative to each otherfor the purposes of assigning grades (e.g., an “A” for students in thetop quartile of the class, a “B” for the students in the middle twoquartiles, etc.) and generating reports on the driving performance forthe class. Further, the driving school employing the driving instructormay use the data and reports to evaluate the skills of the drivinginstructor or the driving school itself. For example, it may beadvantageous to determine whether the classes taught by a particularinstructor demonstrate improved skills over the course of the class. Itmay also be advantageous for the driving school to determine whether theclasses taught at the driving school are effective at improving drivingskills to improve curricula, apply for grants, etc.

Now referring to FIG. 15, another example screenshot 1500 illustrates adriver's education class report from a series of driving sessions. Thescreenshot 1500 may be generated using the techniques discussed inconnection to block 308 and displayed using the techniques discussed inconnection to block 310. In particular, screenshot 1500 shows anumerical tally of the light, moderate, and severe braking eventsrecorded during the driving sessions associated with a particular class(i.e., Tuesday at 8 AM). The screenshot 1500 may list information 1502about the class and a visualization 1504 of the braking event tallies.The visualization allows the instructor and driving school to make aquick evaluation of the performance of each student driver in the class.For example, it is clear from the graph that “Jake” has many morebraking events than “Amy.” Of course, it will be appreciated that anyprimary or secondary indicators or the scores derived therewith may berepresented in a similar manner. For example, a second screenshot 1500(not shown) may display the average speed of each student driver in theclass. If, for example, the second screenshot 1500 (not shown) showsthat “Amy” drove at a very low speed during the driving session, then amore complete comparison of the driving sessions may be made (e.g., Amydid not apply the brake sharply because she was already driving tooslow). Thus, while Amy performed the best in the class at braking, shemay lose points on her grade because of her slow speed relative to theother members of the class.

Of course, it will be understood that the reports displayed on thescreenshots 1100, 1200, 1300, 1400, and 1500 may displayed in ways otherthan being displayed on the screen 202. For example, the reports may beprinted on paper. The reports may also be distributed as discussed aboveusing any suitable technique.

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Additionally, certain embodiments are described herein as includinglogic or a number of routines, subroutines, applications, orinstructions. These may constitute either software (e.g., code embodiedon a machine-readable medium) or hardware. In hardware, the routines,etc., are tangible units capable of performing certain operations andmay be configured or arranged in a certain manner. In exampleembodiments, one or more computer systems (e.g., a standalone, client orserver computer system) or one or more hardware modules of a computersystem (e.g., a processor or a group of processors) may be configured bysoftware (e.g., an application or application portion) as a hardwaremodule that operates to perform certain operations as described herein.

In various embodiments, a hardware module may be implementedmechanically or electronically. For example, a hardware module maycomprise dedicated circuitry or logic that is permanently configured(e.g., as a special-purpose processor, such as a field programmable gatearray (FPGA) or an application-specific integrated circuit (ASIC) toperform certain operations. A hardware module may also compriseprogrammable logic or circuitry (e.g., as encompassed within ageneral-purpose processor or other programmable processor) that istemporarily configured by software to perform certain operations. Itwill be appreciated that the decision to implement a hardware modulemechanically, in dedicated and permanently configured circuitry, or intemporarily configured circuitry (e.g., configured by software) may bedriven by cost and time considerations.

Accordingly, the term “hardware module” should be understood toencompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired), or temporarilyconfigured (e.g., programmed) to operate in a certain manner or toperform certain operations described herein. Considering embodiments inwhich hardware modules are temporarily configured (e.g., programmed),each of the hardware modules need not be configured or instantiated atany one instance in time. For example, where the hardware modulescomprise a general-purpose processor configured using software, thegeneral-purpose processor may be configured as respective differenthardware modules at different times. Software may accordingly configurea processor, for example, to constitute a particular hardware module atone instance of time and to constitute a different hardware module at adifferent instance of time.

Hardware modules can provide information to, and receive informationfrom, other hardware modules. Accordingly, the described hardwaremodules may be regarded as being communicatively coupled. Where multipleof such hardware modules exist contemporaneously, communications may beachieved through signal transmission (e.g., over appropriate circuitsand buses) that connect the hardware modules. In embodiments in whichmultiple hardware modules are configured or instantiated at differenttimes, communications between such hardware modules may be achieved, forexample, through the storage and retrieval of information in memorystructures to which the multiple hardware modules have access. Forexample, one hardware module may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware module may then, at a latertime, access the memory device to retrieve and process the storedoutput. Hardware modules may also initiate communications with input oroutput devices, and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules.

Similarly, the methods or routines described herein may be at leastpartially processor-implemented. For example, at least some of theoperations of a method may be performed by one or more processors orprocessor-implemented hardware modules. The performance of certain ofthe operations may be distributed among the one or more processors, notonly residing within a single machine, but deployed across a number ofmachines. In some example embodiments, the processor or processors maybe located in a single location (e.g., within a home environment, anoffice environment or as a server farm), while in other embodiments theprocessors may be distributed across a number of locations.

The performance of certain of the operations may be distributed amongthe one or more processors, not only residing within a single machine,but deployed across a number of machines. In some example embodiments,the one or more processors or processor-implemented modules may belocated in a single geographic location (e.g., within a homeenvironment, an office environment, or a server farm). In other exampleembodiments, the one or more processors or processor-implemented modulesmay be distributed across a number of geographic locations.

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation.

As used herein any reference to “one embodiment” or “an embodiment”means that a particular element, feature, structure, or characteristicdescribed in connection with the embodiment is included in at least oneembodiment. The appearances of the phrase “in one embodiment” in variousplaces in the specification are not necessarily all referring to thesame embodiment.

Some embodiments may be described using the expression “coupled” and“connected” along with their derivatives. For example, some embodimentsmay be described using the term “coupled” to indicate that two or moreelements are in direct physical or electrical contact. The term“coupled,” however, may also mean that two or more elements are not indirect contact with each other, but yet still co-operate or interactwith each other. The embodiments are not limited in this context.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. For example,a condition A or B is satisfied by any one of the following: A is true(or present) and B is false (or not present), A is false (or notpresent) and B is true (or present), and both A and B are true (orpresent).

In addition, use of the “a” or “an” are employed to describe elementsand components of the embodiments herein. This is done merely forconvenience and to give a general sense of the description. Thisdescription, and the claims that follow, should be read to include oneor at least one and the singular also includes the plural unless it isobvious that it is meant otherwise.

This detailed description is to be construed as exemplary only and doesnot describe every possible embodiment, as describing every possibleembodiment would be impractical, if not impossible. One could implementnumerous alternate embodiments, using either current technology ortechnology developed after the filing date of this application.

What is claimed:
 1. A method comprising: during a driving session inwhich a student driver operates a vehicle while observed by a drivinginstructor: gathering driving skill data indicative of vehicle operationby the student driver from one or more sensors in a mobile device, theone or more sensors including at least one of the following: anaccelerometer array, a geopositioning device, or an image capturedevice, and gathering comments from the driving instructor regardingvehicle operation at one or more geographic locations along a routetraveled by the vehicle during the driving session, wherein the commentsare gathered using a user interface of the mobile device and are eachassociated with one of the one or more geographic locations based upondata gathered from the one or more sensors; generating a driving sessionreport corresponding to the driving session, including: at least onescore based on the gathered driving skill data, and the gatheredcomments from the driving instructor about the driving session; andcausing the driving session report to be displayed on a display devicefor review.
 2. The method of claim 1, further comprising: gathering oneor more additional comments from the driving instructor regardingvehicle operation by the student driver using the user interface,wherein the driving session report further includes the one or moreadditional comments.
 3. The method of claim 1, wherein the gathereddriving skill data indicative of vehicle operation by the student driverincludes indications of one or more of the following: a behavior of thestudent driver during vehicle operation, acceleration of the vehicle,braking of the vehicle, or steering of the vehicle.
 4. The method ofclaim 1, wherein generating the driving session report corresponding tothe driving session includes: calculating a plurality of scorescorresponding to one or more of the following: gaze location, scanningfrequency, lane deviation, lane centering, time to collision, braking,acceleration, or steering of the student driver; and combining theplurality of scores to generate a composite score representative of aperformance of the student driver during the driving session, whereinthe at least one score included in the driving session report includesat least one of the plurality of scores and the composite score.
 5. Themethod of claim 4, wherein at least one of the plurality of scores iscalculated using a weighting factor adjusted for the student driverbased upon previous driving sessions of the student driver.
 6. Themethod of claim 1, wherein: gathering the driving skill data includesgathering a plurality of gaze location log entries using the at leastone image capture device, each of the plurality of gaze location logentries indicating a duration of a gaze of the student driver in aparticular direction.
 7. The method of claim 1, wherein the commentsfrom the driving instructor about the driving session include commenttext written by the driving instructor.
 8. The method of claim 1,wherein the comments from the driving instructor about the drivingsession include at least one audio comment recorded by the drivinginstructor.
 9. A system comprising: one or more processors; a userinterface; one or more sensors, including at least one of the following:an accelerometer array, a geopositioning device, or an image capturedevice; a non-transitory memory storing computer-readable instructionsthat, when executed by the one or more processors, cause the one or moreprocessors to: during a driving session in which a student driveroperates a vehicle while observed by a driving instructor: gatherdriving skill data indicative of vehicle operation by the student driverfrom the one or more sensors, and gather comments from the drivinginstructor regarding vehicle operation at one or more geographiclocations along a route traveled by the vehicle during the drivingsession using the user interface, wherein the comments are eachassociated with one of the one or more geographic locations based upondata gathered from the one or more sensors; generate a driving sessionreport corresponding to the driving session, including: at least onescore based on the gathered driving skill data, and the gatheredcomments from the driving instructor about the driving session; andcausing the driving session report to be displayed on the user interfacefor review.
 10. The system of claim 9, wherein: the non-transitorymemory further stores computer-readable instructions that cause the oneor more processors to gather one or more additional comments from thedriving instructor regarding vehicle operation by the student driverusing the user interface, and the driving session report furtherincludes the one or more additional comments.
 11. The system of claim 9,wherein generating the driving session report further includesgenerating a map, wherein the map includes a representation of each ofthe comments from the driving instructor at the geographic locationcorresponding to each of the comments from the driving instructor, andwherein causing the driving session report to be displayed on the userinterface includes causing the map to be displayed on the userinterface.
 12. The system of claim 9, wherein the computer-readableinstructions further cause the one or more processors to detect one ormore driving events during the driving session based on the drivingskill data, wherein each of the one or more driving events is associatedwith one of the one or more geographic locations along the routetraveled by the vehicle during the driving session.
 13. The system ofclaim 12, wherein generating the driving session report further includesgenerating a map, wherein the map includes a representation of each ofthe one or more driving events at the geographic location correspondingto each of the one or more driving events, and wherein causing thedriving session report to be displayed on the user interface includescausing the map to be displayed on the user interface.
 14. The system ofclaim 9, wherein the driving skill data indicative of vehicle operationby the student driver includes indications of one or more of thefollowing: a behavior of the student driver during vehicle operation,acceleration of the vehicle, braking of the vehicle, or steering of thevehicle.
 15. The system of claim 9, wherein the at least one score basedon the gathered driving skill data is calculated using a weightingfactor adjusted for the student driver based upon previous drivingsessions of the student driver.
 16. The system of claim 9, wherein:gathering the driving skill data includes gathering a plurality of gazelocation log entries using the at least one image capture device, eachof the plurality of gaze location log entries indicating a duration of agaze of the student driver in a particular direction.
 17. The system ofclaim 9, wherein the at least one score includes an acceleration,braking, or steering score based on first portions of the driving skilldata generated by the accelerometer and a gaze location, scanningfrequency, lane deviation, lane centering, or time to collision scorebased on second portions of the driving skill data generated by theimage capture device.
 18. A non-transitory computer-readable mediumstoring instructions that, when executed by one or more processors of acomputer system, cause the computer system to: during a driving sessionin which a student driver operates a vehicle while observed by a drivinginstructor: gather driving skill data indicative of vehicle operation bythe student driver from one or more sensors in the computer system, theone or more sensors including at least one of the following: anaccelerometer array, a geopositioning device, or an image capturedevice, and gather comments from the driving instructor regardingvehicle operation at one or more geographic locations along a routetraveled by the vehicle during the driving session, wherein the commentsare gathered using a user interface of the computer system and are eachassociated with one of the one or more geographic locations based upondata gathered from the one or more sensors; generate a driving sessionreport corresponding to the driving session, including: at least onescore based on the gathered driving skill data, and the gatheredcomments from the driving instructor about the driving session; andcause the driving session report to be displayed on a display device forreview.
 19. The non-transitory computer-readable medium of claim 18,further comprising instructions that cause the computer system to gatherone or more additional comments from the driving instructor regardingvehicle operation by the student driver using the user interface,wherein the driving session report includes the one or more additionalcomments.
 20. The system of claim 18, wherein the at least one scorebased on the gathered driving skill data is calculated using a weightingfactor adjusted for the student driver based upon previous drivingsessions of the student driver.